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ABSTRACT ' 

A procedure is outlined for understanding and 
evaluating interagency collaboration. Fivb steps are addressed: (1) 
understanding the context of an interagency collaboration (stages of 
development, communication channels used); (2) verifying the need for 
interagency collaboration (needs for increased client access to 
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EVALUATI NG * INTERAGENCY - COLLABOI^TIONS 



Interagency collaboration is «n active process, and efforts to evaluate 
it must reflect its dynainiQ, procedural chara<;teri sties. Evaluation studies 
should examine the Intent of a collaboration- and may focus on one, some, or 
all of the stages of a collaborative program. Any evaluation Istudy should^W 
designed and implemented to yield information that will satisfy needs of those 
directing the collajjorative effort. 

This paper will discuss the role of evaluation as members of an inter- 
agency planning team attempt to; . 

• Understand/the context of an Interagency collaboration 

• Verify the need for the collaboration 

• Identify solutions and plans to meet needs 

• . Implement the collaborative process 

• Determine the outcomes (benefits and liabilities) of the collaboration 

UNDERSTANDING THE CONTEXT 



Four kinds of information will help you understand the context in which 
interagency collaboration takes place : 

• The particular developmental stage of the collaboration; 

• A description of the agencies and people involved; 

• A definition of the terms used to describe the collaboration; 

• A description of the channels of communication used In the 

• collaboration* ' * 



Stage of Development 

» • 

There are many models or frameworks for the development of Interagency 
collaborative efforts* These set forth the basic life cycle of the collabora- 
tion (Figure 1 shows typical flow of eventsO 

To begin an evaluation of Interagency collaboration^ detennlne the current 



. Figure .1 ! . " 

A Process Outline for Interagency Planning^ 



Establish the Need 


Determine needs and rationale or 




Define service delivery population 




for Initiation of Interprogram 




of interest. J . 




collaboration project • 











Establish the Data 
Base 



Identify the Planning 
Targets 



Establish Interagency 
Provisions 



Assure Collaboration 
In Service Delivery 



Identify' agencies and progr^ips 
serving or authorized to serVe 
the target p6pulatlon(s) aivd 
cdntact agency administrator. 




0 

Define current program policies and 
services and respQnslbllltles of 
Identified programs. 




J 



Compare local programs and proce- 
dures across agencies to Identify 
gaps, overlaps, constraints, and 
other linkages. 




Identify local policy and procedures 
wherein modifications would enable 
satisfaction of need and rationale 
for collaboration and specify the 
needed modifications. 







DetermlnjB which modifications can 
be made on the local level and 
incorporate these modifications 
in a local interprognam agreement < 
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Enable implementation of inter- 
program modifications. 



Implement local evaluation functions. 



^Regional Resource Center Task Force > 1979. 
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state of the collaboration in relation to its life cycle • You may start By 
interviewing key personnel among the providers, administrators, business 
managers, and consumers. within the agencies involved in the collaboration. 
Questions addressed in the ihterview could center on the stages identified in 
Figure 1. Further, the evaluator could conduct a record review to obtain 
information which would support the testimony of those interviewed and extend 
the evaluator 's understanding of ^ the program. 

Another approach which is decidedly more complex and time consuming, but 
which may provide more Information, focuses on the process of adopting an 
^innovation.. If an interagency collaboration is thought of as an innovative 
approach to service delivery, we can then view its adoption much like an adop- 
tion of an innovative classroom. practice. The adoption is not a* single event; 
rather, it is a process. 

Gene Hall and his colleagues (University of Texas at Austin, Research and 
Development Center for Teacher Education) developed a- framework to assess the 
adoption of educational innovatioi? (Hall, 1975). McLaughlin and Elder ^ 
(1982) translated that framework so that it might be used to determine the 
opjer^tional stage of an interagency collaborative effort. The translated 
stikges are set forth in Figure 2. To determine the stage for a particular 
interagency effort, evaluation is focused on the following factors or beh^avj^ors 
associated with people engaged in the effort: , 

• knowledge of interagency collaboration; 

• acquiring information about collaboration; ^ 
sharing information within and across agencies; 

• assessing/evaluating the agreement or need for agreement; 

• status reporting relative to the agreement; / 

• planning related to interagency /collaboration; 

• performance of specific Individuals as they carry out the agreement. 
Concern about the collaboration expressed by* the people involved is 

another factor which may be considered in order to determine the current oper- 
ational stage of the cfbllaborative program.. Again, we have taken from the work 
of Hall and others. Here we have tried to develop a scale whii^h describes 
levels ojf concern (sefe Figure 3). There ate three major levels of concern: 
those unrelated to the collaboration; those oHented to self i^nd the collabo- 
ration; and those related to the, degree to which the collaboration is meeting 
the Aeeds of the client* In our opinion, these levels are directly related to 
the developmental stages of the collaboration. ^ % 

Understanding the levels of concern of those involved in the collaboration 
and knowledge of the stages of development through which a program normally 
progresses can be a useful combination of information for planners who are try- 
ing to decide where to go next. If the intetagenoy effort is at a standstill 
(Figure 4 i^hows some common problems), then the inform^^tion acquired by the 
evalOator can be use^'to pinpoint technical assistah^e which will facilitate 
movement to the next higher stage of development. 

To recap: to begin an evaluation of an interagency collaboration, deter- 
mine, the particular" progr^m^s current point In the life cycle of a typical col^ 
laboration. , Investigation may focus on the collaboration process, the adoption 
(of an it)novat;ion) process, or the levels of concern of the people involved in 
the program. Multiple datd-gathering strategies and points of focus are recom-* 
mended: conduct interviews;, review records, reports, and minuted of meetings; 
and' use surveys with Interviews to confirm and extend findings. The product of 
this effort will be an understanding of the development^al stage and a deter- 
mination of any variables in development across agencies. 
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Figure 2 / 

DESCRIPTION OF LEVELS OF USE FOR INTERAGENCY COLLABORATION^ 



LEVEL 
0: Non-use t 



I: 



Orientation. 



XI: Preparation. 

J 

III, Mechanical Use* 



IV-A: Routine, 



IV-B: Refinement. 



Integration. 



VI: Renewal. 



DESCRIPTION 

State In which the user has little or no knowledge - 
of Interagency collaboration^ no Involvement with 
Itt and Is doing nothing to become Involved. 

State In which the user has recently acquired or Is 
acquiring Information about Interagency collatiiora- 
tlon and/or has recently explored or Is exploring 
Its value; and Its demands. 

State In which the user Is preparing for Implementa- 
tion of Interagency collaboration. 

State In which the user focuses most effort on the 
short-term> day*"to^ay use of Interagency collabo- 
rative efforts with little time for reflection. 
Changes In use are made more to meet user needs than 
client needs. The user Is prlmatl^lV^ engaged In a 
step-by-step attempt to master th^^ tastes required to 
Implement the collaboration; this often results In 
disjointed and superficial use. 

Use of Interagency collaboration Is stabilized. Few 
If any changes are being made In ongoing use. Little 
preparation or thought Is being given to Improving 
collaborative efforts or Its consequence^ . 

State in which the user varies the use of Inter- 
agency collaboration to Increase the Impact on 
clients within Immediate sphere of Influence. Vari- 
ations in collaborative atrangements are based on 
knowledge of both 6hort- and lojig-term consequences 
for cli'fijrits. 

State In which the user Is combining own efforts to 
use Interagency collaboration with ^elated activi- 
ties of colleaguias to achieve a collective Impact on 
clients within the community. 

State In which the user reeva]^uates the quality of 
use of Interagency dollaboratlon^ seeks tea j or modi- 
fications or alternatives to achieve Increased 
Impact on clients; examines new developments In 
the field and explotes new goals for self and thilf^ 
collaborative service delivery system. | 
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^McLaughlin and Elder, 1982. 
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Figure 3 

LEVELS OP CONCERN IN AN INTERAGENCY COLLABORATION^ 

• , t. , 
I. Concerns About Self 

0. Non-lnteragency concerns. 

II. Concerns About Self as a Member of the Collaboration Te^ 

1. Where do I stand in relation to the collaboration team? 

2. Do I functjon adequately in the collaboration? v 
^ 3. How do the^thers in' the collaboration view me? ' 

III. Concerns About the Interagency -Coljjaboratlon 

4. Are the clients getting what we want them to get? Is the 
collaboration doing what we want it to do? 

5. Is the collaboration meeting the needs of the targets? 

6. How can the collaboration be improved? 

McLaughlin and Elder, 1982. 



Figure k 



POTENTIAL PROBLEMS ASSOCIATED WITH ESTABI^ISHING 
INTERAGENCY COLLABORATIONS 



Competitiveness of established institutions and agencies. 

• Lack of an organizational structure that brings agencies together 
V around mutual Interest. 

• Parochial interest of agencies that make ..them .norppic to the needs 
of the broader community. 

• Lack of experience in the techniques of coordinating service 
delivery. ^ 

•^.Awkward interdisciplinary communication. 

• Preoccupation with the system design ra^ther than the functional ^ 
role of the system. 

• Client confidentiality across agencies. 

• Personal resistance to change. . . 

• Response to external pressures. 

• Lack of specific accountability. « 

• Lack of designated monitors and evaluators. 

• Inadeqjwtte orientation within and* outside agencies. - 

• Negative staff attitude* > 

• Lack of conslde^tion of political bases.. 
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De8cr:^ptlon of ^gencles * ' * 

' As netfds are Identified and potential solutions sought. It will be neces^ 
sary to under^stand each agency and Its functional relationship to the needs. 
Also^ we we have found that successful planning Is more likely if agencies are 
given a chance to completely describe their services and functions. 'However, a 
profile of this sort should not «be the only means by which agencies find out 
about each oi\her. Time must be allotted for all Involved in the collaboration 
to discuss a^d visit each agency* 

The Regional Resource Center Task Force .on Interagency Collaboration 
(1979) Suggested that the following Information be contained In an agency 
profile: ^ V • ♦ 

• Program Overview " . . 

— legislative base (federal/state) 
administrative agency (federal/state) 
general purpose of agency 

— client eligibility 

— application procedures 

— type of funding \ 

— assurances/t)rocedural safeguards ^ 

— monitoring responsibility , 

— sanctions authorized 

— policy makers i 

• Client-Centered Services ^ " ' ^ 

— Initial client Identification (oUtreAch/screenlng) / 
comprehensive mul tldlsqlpllnary assessment 

— Individual client pl^n development ^3 

— placement, options (treatment) ♦ 
service models * 

— service review (follow-up evaluation) 

• Support Processes * 

: material support* (facilities) ^ ^ 

— personnel support (certification requirements, In-servlce opportuni- 
ties, staffing patterns, etc.) 

— fiscal support (patterns of allocation across agency functions) 

— planning bodies (advisory groups) 



Definition of Terms 

r J . 

/. . ' « 

•4 

Jargon q^ten contributes to conflicts in the planning and implementation 
of a collaborative activity. So, as part of the profile, develop a glossary of 
terms commonly used by agency staff. ^ gl^ssaryvhelpe participants as they 
review othet agencies' documents and interact with each other. A comparison of 
the glossaries T>f all agencies will help all participants develop a common set 
of reference terms. ^ 



Chantielg of Communica t ion * 

Who talks to whom? What formal ot infottaal vehicles are used to communi- 
cate? Whafc types of Information ^low through the channels? Have rules been 
established for th6 control and f iow of Information? 

Ask participants to construct existing and desired communication paths by 
graphically depicting the flow of information. Discrepancies between i^xlsting 
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and desired paths can Ijad to, change which will Improve the collaborative 
effort. The desired path of comnwnlcatlon can also serve as a standard for 
future evaluation. - ^ . 



VKRIFypfi THE NEED FOR INTERAGENCY COLLABORATION 

. • ■; * * « 

^search hae^^jlndlcated that the following problems are generally 
isons for an interagency collaboration: 
fragmented service dellNery system 

• .overlap in service definitions < 

• multiple funding bases . ' * . . 

• varying models for service delivery (e.g., medical or educational) 

, Fragmented service delivery systems can cause clients to "f^ll between the 
tracks" as they try to move from one service to the next. Cent ra^l4^ed client 
management systems can help. In* such cases, you may need to gather data sVb- 
stantiating that there are gaps in the system. Interviews and surveys of 
sumers and key people in agencies and 'advocacy groups can be useful. 
Review of rejjords (including consumer complaints or letters of support) a.f^ 
also helpful. , 

Try developing hypothetical case profiles and nfovlng them through the syt 
tem from agency to agency. Specifically, ask representatives from each agency^ 
to review the profile and talk through the way the^f agency would approach the 
problem from referral to evaluation to planning to delivery of services. Com^ 
parison of the responses can indicate problems in the current service system 
and problems that might arise if the agreement is established in manner con^ 
tradictory to existent-^practlce . 

Often, when service definitions of different agencies overlap, resources 
are wasted as va^rious figencies work to provide the same services toTYie saiqe ' 
clients. Here it may be useful to identify service responsibilities and func- 
tions which are common and unique to particular agencies. Interviews }fkth key 
agency staff members and a review of agency mission statements and regulations 
will help identify servicfes. * 

Multiple fuhdingibases and multiple planning bodies tend to drive agencies 
in different direction!. That is, funds are usually tied to regulations which 
may or may not be comnton across agencies. Fiscal t:alendars may differ*. Docu- 
mentation of the types of funding and the purposes of the planning bodies can 
help interagency planners. Review of agency regulations and mission statements 
is imperative. ^ 

Finally, variations in models for service delivery may be a barrier to 
an interagency collaboration. Personnel within different agencies often have 
different approaches to solving client problems. Interviews arid the hypo- 
thetical case technique can help determine if variations in approach exist 
across agencies. . 

The common thread that runs through the four problems set forth abov^ ;ts 
increased client access to services and reduced duplication of services. (Note 
th^ sometimes conditions exist — e.g., large jiumber of clients, large gpo- 
graphic area, etc. — which require that various agfencles provide similar ser- 
vices.) , . 



The Human Factqr , ^ ' * J 

^ Human factors play an lijbportant part^^^ thf success or failure of an 
Interagency mission. Some aithors havje ca¥t;\thebe human factors under the 
domain of team life (Magrabylno date). An evif^luatloti must constantly examine ^ 
members of the team. Magrab has developed SQnf^ simple, questionnaires ho accom- 
plish that task. Things to look for are: lea'deirshlp, noXe clarification^, and 
'group* atmosphere. Personality conflicts can be most dangerous to group cohe- * 
slon and accompllstVii^ent of a mission. * 

The ^evaluation must take Int^^ account the squrces of conflict^ In the grouj) 
(see Figure 5). People. manage conflict ^i^n a number of ways. Som^ people come 
on strong and put themselves In a win-lose situation;, others avoid or withdraw < 
from conflict; some will try to compromise oEsnegotlate to alleviate conflict • 



Figure 5 

SOURCES OF CONFLICT ft INTERAGENCY COLLABdRATION 
The People ; ' The Environment ; 



Value differences: 

— personal 
--professional 

— agency x 
Rple pressures 
Perceptual differences 
Divergent goals 
Status threat : 
Personality clash 



• Lack of resour^bes 

• Change 

% Ambiguous Jurisdictions' 

• Communication barriers: . - 

— personal 

— physical 

• Dependence on one parson/agency 
for agreement 

Complexity of organlzatlonCs) 

• ileed f^r consensus 

• Regulations 



Each Interagency planning team Is likely to have members who-«*ie each of ^ 
the 'ab^ve styles under certaln conditions* If managed appropriately, con*- 
fllctHE^n daiuse the group to become creative and look for new and Innovative 
approaches to their problems. Evaluation can Identify appropriate and inap- 
propriate ^nflltzt management, and strategies for communicating to the members 
the effects of theif conflict styles* 
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IDENTIFYING SOLUTIONS AND PLANS TO MEET, NEEDS 




An lijb«ragency i>latt Will only succeed if those responsible for adniltll8-• 
te^lIlg^ and dellyeci^^^ to the, need for the Qollabo- 
ratipn and lb the implem&atioh of the plan. ThW suggests that an evaluation 
d^nclii^e aoa^< early and periodic proceidures that will assess comi^ltment * 
^ •••'^^^ shading of information .lift essential. Member^ of 
fti*^iftte<sM;agency keep , people within eifch agency apprilspd of 
the' stattt* <*f ali%i*tt*i^^^^^^^^^^^^^^ include all persons 



potentially affected by the agreement : .administrator^^ , providers^ business 
managers, and consumers. > ' - / 

. According to Elder (1980)^ when a plan or paper agreement Is 'completed^ It 
should have the following component 6: ' ' 

• Clear statement of purpose with goals and measurably objectives. 

• Definition of termd central to the operation of the agreement. 
Precise dellneatit)rrof services to be provided by each agency. 

• A statement of fiscal repponsiblli ty (if required) for^each service 
provided. 

• Designation of roles and responsibilities (within and Across agencies) 
associated with each service or collaborative activity. • f 

• Designation of ^staf f positions within each agency responsible for: 

- Implementing the service 

- monitoring the implementation 

- negotiating change when necessary. ^ . • 

• Specification of general administration procedures including: 
• - scheduled meetings 

- time period for the agreement 

- client tuanagement 

- mechanisms for change. 

• Evaluation plan for determining the extent to which the agreement's 
short- and long-term objectives/are met. • * 

All this will look Just fiuQ on, paper and will serve as an excellent 
implementation standard for both the project and the evaluation. However, 

tthe plan or agreement is put into practice » we suggest an evaluation 
gy that examines the logical, technical, and political structure of the 
(Here, politics refer to personal interactions surrounding issues of 
) 

Data that will help you determine if your plan is sound can be acquired by 
sending copies of the written plan togethe^r with a structured rating form to 
four paifties; the planners, agency administrators^ representatives of agency 
providers affect;*ed by the plan» and representatives of consumer groups. The 
primary questions are:, 1) Are purposes, goals, and objectives appropriate? 
and 2) Will these be achieved given the plan's construction? 



IMPLEMENTING THE COLIABORATIVE PROCESS 



During the implementation phase of the collaboration, the evaj|.uation is^ 
guided by the plan discussed above. As with any evaluation, two* maior ques- 
tions must be addressed by the program misinagers and evaluators: l)^l6 the plan 
Implemented according to design?- and 2) Are the objectives of the agreement 
being met? A clear plan with clear objectives will provide the standards.^ 

At this point, monitoring the degree to which the agreement is imple-- 
mente4 as designed becomes a critical component of the evaluation. Data must 
be derived which provide evidence pf what actually is taking place with regard 
to the delivery of service to the target clients*^. A mls^tdke often made at this 
point in the evaluation Is to only gather outcome data. When this occut'St all 
that can be said is: "You have (have not) accomplished this." In order to 
explain why things turned out as they did; we suggest the development of an ' 
Implementation checklist associated with each compdnent of the plan. Data 



regarding the level of Implementation may be obtained through Intervlewa^^ 
record review, or both. As a result of this type of evalua^Cflon , collabor- 
ators can know why something occurred and they can have some suggestions as 
to what wllL occur In the ^u^j^pe If'i ^thanges arep^t made. 

Evaluation of accomplishments Is also Important; we want to knov what we 
are doing In relation to the agreement and^what Impacts are being made. Are 
^ervlceef more accessible? A^^ costs being reduced? is there less duplication 
of services across agenci^es?^ Xre more .(or fewer* clients beln^ served by a 
partlculj^r agency? Is there a single case otanagement function .being performed? 
Is there common planning? 

The specific .questions to be addressed at this point of^ the evaluation 
depend on the purposes of the, agreement • However, your plan must be flexible 
enough to capture unintended or unexpected outcomes. Monitoring and outcome 
evaluation concerns will take their dlrec;tlQn from the colldboratlon plan. Be 
prepared to .put both types of data together In order to explain events to deci- 
sion makers. * 



PETER>lijNING THE BENEFITS AND LIABILITIES OF THE COLLABORATION . 

» 

Most Interagency collaborative efforts do. not have a termination date, so 
there Is no specific time at which the planners can sit back and look at their 
accomplishments. When this Is the case, outcomes become difficult to gaifge. In 
the evaluation literature, many experts advise that summatlve Judgments of pro- 
grams not be attempted until the programs reach some degree of stability. 
Since Interagency collaborative efforts are often In a state of constant flux, 
summatlve judgments. In the strict sense of the concept, may be Inappropriate. 
Instead, you can periodically focus your efforts on the status of the benefits' 
the collaboration may produce for the participating agencies and their clients 
(see Figure 6). 



Figure 6 

PROJECTED BENEFITS FROM THE ESTABLISHMENT OF 
INTERAGENCY COLLABORATIVE AGREEMENTS 



• More accessible services ^. 

• Common program stan^rds and uniform methods of accountability 

• Single responsibility for case management 

• Cooperative Identification, evaluation, planning, and service delivery 

• An Inventory of service capacity at state and local levels 

• Clarification of reopongiblWLty for fiscal support 

• Common planning ^ 

• Reduction in redundant bervic^s 

• Reduction in the total cost of services for persons with handicaps 



f . . CLOSING 

. . ■ • ■ 

As indicated aC Che start of this paper^ the* evaluation of an Interagency 
collaborative effort depends on the Intent of the effort. Therefore, you and 
your evdluator must determine the audiences, both Internal and external to the 
program, to Identify their Information needs and , expectations. The Information 
gleaned from the ^valuation can best be used" In the formative sense — • to siig-^' 
gest changes^ In the current system that will Increase accessibility to needed 
Services at the le^^st human a^d fiscal costs. 
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